Image processing apparatus and image processing method

ABSTRACT

An image processing apparatus includes an acquiring unit, a specifying unit, a smoothing unit, and an output unit. The acquiring unit acquires resolutions of first and second projectors and region information regarding projected regions. The specifying unit specifies a boundary between the projected regions, the second projector having higher resolution than the first projector, based on the acquired region information. The smoothing unit executes smoothing on a predetermined region within a projected region based on the acquired resolutions of the first and second projectors such that a pixel, obtained based on a first pixel by executing the smoothing, has been affected according to a second pixel then a third pixel, wherein the first pixel is in the predetermined region, the first and second pixels are arrayed parallel to the specified boundary, and the first and third pixels are arrayed perpendicular to the boundary. The output unit outputs smoothed image data.

TECHNICAL FIELD

The present invention relates to an image processing method when animage is projected by using a plurality of projectors.

BACKGROUND ART

Recently, multi-projection has been widely used in which an image basedon one set of image data is displayed by using a plurality ofprojectors. In the following, a manner of displaying one image byarraying projected regions of the plurality of projectors in a tiledpattern is particularly called “tiled display”.

As a method of coupling adjacent projected regions of the projectors toeach other in the tiled display, the following examples are known. Asone example, there is known a method of coupling two projected regions Aand B while making adjustment such that those projected regions are notoverlapped with each other. As another example, there is known a methodof preparing an overlapped region at the boundary between the projectedregions A and B, projecting a common image P to each of respectiveoverlapped regions, and coupling those overlapped regions while makingadjustment such that the common images P are exactly superposed witheach other. Patent Literature 1 discloses a method of automaticallydetermining a start point and an end point of an overlapped region (edgeblend) at each boundary between respective projected regions of theplurality of projectors.

CITATION LIST Patent Literature

-   PTL 1 Japanese Patent Laid-Open No. 11-98439

However, an image based on image data is displayed by using a pluralityof projectors, there is a risk that the boundary between respectiveprojected regions of the projectors is conspicuous.

For example, when the tiled display is performed by using a plurality ofprojectors having different resolutions, there is a risk that theboundary between the projector having a lower resolution and theprojector having a higher resolution is more conspicuous than the casewhere the tiled display is performed by using projectors having the sameresolution.

As another example, when, of first and second projectors having the sameresolution, a field angle of the first projector is set to be smallerthan that of the second projector and the first projector projects animage within a projected region of the second projector, there is a riskthat the boundary between the respective projected regions isconspicuous. This is because, when the projectors having the sameresolution are used, the projector having a smaller field angle providesa higher resolution and hence a difference in resolution is conspicuousat the boundary between the respective projected regions.

Further, for example, when the tiled display is performed by using aplurality of projectors in which keystone correction is made forrespective image signals, there is also a risk that a difference inresolution is generated at the boundary between respective projectedregions of the projectors and the boundary is conspicuous.

When the boundary is conspicuous due to the difference in projectorresolution, the boundary can be made less conspicuous, for example, byreducing the resolution of the projector having the higher resolution.However, image quality in the projected region of the projector havingthe higher resolution degrades. As another example of solution, theboundary can be made less conspicuous by reducing the resolution only ina part of the projected region of the projector having the higherresolution, which is positioned near the boundary. However, imagequality degrades near the boundary.

The present invention has been accomplished in view of the problemsdescribed above, and its object is to make the boundary betweenprojected regions less conspicuous while reducing degradation of imagequality when an image based on image data is displayed by using aplurality of projectors.

SUMMARY OF INVENTION

To solve the problems described above, an image processing apparatus ofthe present invention is constituted, by way of example, as follows. Inan image processing apparatus for causing a plurality of projectors toproject an image, the image processing apparatus includes an acquiringmeans, a specifying means, a smoothing means, and an output means. Theacquiring means is for acquiring resolutions of the plurality ofprojectors and region information regarding projected regions of theplurality of projectors. The specifying means is for specifying aboundary between the projected regions of the projectors havingdifferent resolutions based on the resolutions of the plurality ofprojectors and the region information, which have been acquired by theacquiring means. The smoothing means is for executing smoothing suchthat pixels arrayed in a direction parallel to the boundary, which hasbeen specified by the specifying means, are more strongly smoothed thanpixels arrayed in a direction perpendicular to the boundary. Inaddition, the output means is for outputting image data that has beensmoothed by the smoothing means.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments with reference to theattached drawings.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a block diagram representing a multi-projection systemaccording to Embodiment 1.

FIG. 2 is a block diagram of an image processing portion 201 inEmbodiment 1.

FIG. 3 is a flowchart representing processing executed in an imageprocessing apparatus 101 in Embodiment 1.

FIGS. 4A and 4B are each an illustration to explain a boundary regionand a filtering process in Embodiment 1.

FIGS. 5A to 5E are each an illustration to explain the filtering processin Embodiment 1.

FIGS. 6A and 6B are each an illustration to explain pixel shapes in thefiltering process in Embodiment 1.

FIG. 7 is a block diagram representing a multi-projection systemaccording to Embodiment 2.

FIG. 8 is a block diagram of an image processing portion 1201 inEmbodiment 2.

FIGS. 9A to 9C are each an illustration to explain division of an image,which is executed by an image dividing portion in Embodiment 2.

FIGS. 10A and 10B are each an illustration representing a tangentialline and an angle with respect to a boundary region in Embodiment 2.

DESCRIPTION OF EMBODIMENTS Embodiment 1

The configuration for multi-projection in this embodiment will bedescribed with reference to FIG. 1. This embodiment is described inconnection with an example in which an image processing apparatus 101causes two projectors A and B to perform tiled display.

The projector A is constituted by a panel signal generating portion 102,a light source 103, a liquid crystal panel 104, and a projection opticalsystem 105. The panel signal generating portion 102 executes imageprocessing on image data sent from the image processing apparatus 101 togenerate control signals and outputs the control signals to the liquidcrystal panel 104. The liquid crystal panel 104 is constituted by threepanels of R/G/B, and their transmittances for the three primary colorsR/G/B are two-dimensionally controlled in accordance with the controlsignals from the panel signal generating portion 102. Light projectedfrom the light source 103 passes through the liquid crystal panel 104that is controlled in accordance with the control signals generated bythe panel signal generating portion 102.

The transmitted light having passed through the liquid crystal panel 104passes through the projection optical system 105 and is projected onto ascreen. The projection optical system 105 is constituted by a lens forprojecting an image onto the screen in accordance with the transmittedlight from the liquid crystal panel 104, a zoom portion for changing afield angle on the screen, a lens shift portion for changing a displayposition on the screen, etc. While the projector B has a similarconfiguration to that of the projector A, the projector B has aresolution three times higher than that of the projector A. Accordingly,the resolution to be controlled in a panel signal generating portion 106and a liquid crystal panel 107 of the projector B is higher than that inthe panel signal generating portion 102 and the liquid crystal panel 107of the projector A.

Next, the image processing apparatus 101 according to this embodiment isdescribed. The image processing portion 201 is a circuit for receivingthe image data, executing image processing (filtering process) for theboundary between the respective projected regions of the projectors usedfor the tiled display, and outputting the image data after the filteringprocess to the projectors. As an alternative, a CPU 202 included in theimage processing apparatus 101 may read a program, which is recorded ina ROM 203, into a RAM 204 and may execute the above-mentioned filteringprocess, as required. In such a case, the ROM 203 and the RAM 204provide programs, data, and work areas, which are necessary to executethe filtering process in this embodiment, to the CPU 202. The filteringprocess executed by the image processing apparatus 101 according to thisembodiment is a smoothing process for smoothing pixel values within afiltering region. The smoothing process in this embodiment is executedby using a mean value filter. However, the smoothing process may beexecuted by using a higher-frequency component cut filter for cuttinghigher-frequency components in the filtering region, or a central valuefilter that employs a pixel value at a center of the filtering region.

Also, the image processing apparatus 101 includes a manipulating portion205 for inputting an instruction from a user. The manipulating portion205 can be realized with buttons or a touch panel, for example. Theimage processing apparatus 101 further includes an external interface206 for connection to a PC, a camera, and media (such as a hard disk, amemory card, an SD card, and a USB memory). The external interface 206can be realized with, e.g., a communication line in conformity withstandards, such as USB, LAN or IEEE1394, or realized with wirelesscommunication.

The image processing portion 201 in this embodiment executes thefiltering process (smoothing process) to make the boundary betweenrespective projected regions of the projectors having differentresolutions less conspicuous by utilizing visual characteristics of theuser. More specifically, there are such visual characteristics that theresolution difference in the direction parallel to a boundary linebetween two regions differing in resolution from each other more greatlycontributes to recognition of the boundary line than the resolutiondifference in the direction perpendicular to the boundary line. Inconsideration of those visual characteristics, the image processingportion 201 executes the filtering process (smoothing process) of theimage data such that pixels arrayed in the direction parallel to theboundary between the projected regions of the projectors havingdifferent resolutions are more strongly smoothed than pixels arrayed inthe direction perpendicular to the boundary therebetween. That type offiltering process is executed as a process of, with the aid of a filter,reducing the resolution of the image data, which is output to theprojector having the higher resolution (i.e., the projector B), to bematched with the resolution of the projector A having the lowerresolution. Thus, the image processing portion 201 in this embodimentexecutes the filtering process for the pixels arrayed in the directionthat does not greatly contribute to recognition of the boundary line(i.e., in the direction perpendicular to the boundary line), at a weakerlevel than the filtering process for the pixels arrayed in the directionthat greatly contributes to recognition of the boundary line (i.e., inthe direction parallel to the boundary line). As a result, the boundarybetween the projected regions of the projectors can be made lessconspicuous while suppressing degradation of image quality.

Details of the image processing portion 201 are illustrated in FIG. 2.The image processing portion 201 includes a parameter acquiring portion301, an image dividing portion 302, a region specifying portion 303, acoefficient determining portion 304, and a filtering portion 305. Theparameter acquiring portion 301 acquires region information regardingrespective positions of the projected regions of the projectors used forthe tiled display, and respective resolutions of the projectors used forthe tiled display.

The image dividing portion 302 divides the image data, which has beeninput to the image processing apparatus 101, based on the regioninformation of each projector. The region specifying portion 303specifies the boundary between the projected regions of the projectorshaving different resolutions based on the region information and theresolution of each projector, which have been acquired by the parameteracquiring portion 301, and then determines, based on the specifiedresult, the filtering region where the filtering process is to beexecuted.

The coefficient determining portion 304 determines a filter coefficient(smoothing parameter) used in the filtering process. The filteringportion 305 executes the filtering process for the filtering region,which has been specified by the region specifying portion 303, inaccordance with the filter coefficient (smoothing parameter) determinedby the coefficient determining portion 304.

The operation of the image processing apparatus 201 will be describedbelow with reference to a flowchart of FIG. 3.

In step S301 (acquisition procedure), the parameter acquiring portion301 acquires the region information of each of the projectors used forthe tiled display. The region information implies information forspecifying the positional relationship between respective projectedregions of the projectors in the multi-projection. In the case of thetiled display illustrated in FIG. 1, for example, the region informationacquired by the parameter acquiring portion 301 is given as informationrepresenting that, in the configuration in which the projected regionsof two projectors are arrayed side by side, the projector A projects animage to the projected region on the left side and the projector Bprojects an image to the projected region on the right side. Further,the region information includes information regarding the size of theprojected region of each projector. The parameter acquiring portion 301in this embodiment acquires the region information based on details ofsetting, which have been set by the user through the manipulatingportion 205.

For example, the user inputs the number of projectors used for the tileddisplay and selects a configuration pattern of the tiled display.Further, the user designates the projector for each of the projectedregions corresponding to the selected configuration pattern. Theparameter acquiring portion 301 acquires the region information based onboth the configuration pattern of the tiled display, which has beenselected by the user, and the projector designated for each of theprojected regions.

In the case of FIG. 1, for example, “2” is input as the number ofprojectors, and a configuration pattern in which the projected regionsare arrayed side by side is selected as the configuration pattern of thetiled display. Further, in the configuration pattern in which theprojected regions are arrayed side by side, the projector A isdesignated as the projector that projects an image to the projectedregion on the left side, and the projector B is designated as theprojector that projects an image to the projected region on the rightside. The parameter acquiring portion 301 in this embodiment acquiresthe region information based on the above-mentioned user inputs(designation information inputs).

Stated another way, the parameter acquiring portion 301 acquires theregion information regarding the projected regions of the pluralprojectors based on the designation information input through themanipulating portion 205.

As an alternative, the parameter acquiring portion 301 in thisembodiment may acquire the region information based on an image pickedup by a CCD sensor camera that is connected to the image processingapparatus through the external interface 206. More specifically, the CCDsensor camera picks up respective images of the projected regions of theprojectors. By receiving the images picked up by the CCD sensor camera,the parameter acquiring portion 301 can acquire the region informationregarding the projected regions of the plural projectors. With such amodification, the region information can be acquired even when the userdoes not input the configuration pattern of the tiled display. Inaddition, the parameter acquiring portion 301 in this embodiment changesa manner of acquiring the region information so as to acquire the regioninformation based on the picked-up image when the region informationcannot be acquired from the user inputs. As a result, the regioninformation can be acquired with higher reliability.

In step S302 (acquisition procedure), the parameter acquiring portion301 acquires respective resolutions of the projectors A and B. Theparameter acquiring portion 301 in this embodiment receives noticesindicating the respective resolutions from the projectors A and B viadisplay cables. The parameter acquiring portion 301 acquires therespective resolutions of the plural projectors based on the noticessent from the plural projectors.

Further, the parameter acquiring portion 301 in this embodiment maycalculate the resolution of each projector from the image picked up bythe camera that is connected to the image processing apparatus throughthe external interface 206. With such a modification, the parameteracquiring portion 301 can acquire the resolution of each projector evenwhen the projector does not have the function of notifying itsresolution. In addition, the parameter acquiring portion 301 in thisembodiment changes a manner of acquiring the resolution so as to acquirethe resolution based on the picked-up image when the resolution cannotbe acquired based on the notice sent from the projector. As a result,the resolution of each projector can be acquired with higherreliability.

In step S303, the image dividing portion 302 divides the image databased on the region information of the projectors, which has beenacquired by the parameter acquiring portion 301. In this embodiment, asillustrated in FIG. 1, the projected regions of the projectors A and Bare arrayed side by side in the left-and-right direction so as toperform the tiled display, and the projected regions have the same size.Therefore, the image dividing portion 302 horizontally divides the inputimage data into the same size as viewed in the left-and-right direction.It is here assumed that the projected regions of the projectors A and Bare coupled in a state not overlapping with each other.

In step S304 (specifying procedure), the region specifying portion 303specifies the filtering region based on the region information and theresolutions, which have been acquired by the parameter acquiring portion301. More specifically, in step S304, the region specifying portion 303specifies the boundary between the projected regions of the projectorshaving different resolutions based on the resolutions and the regioninformation of the plural projectors. Further, the region specifyingportion 303 specifies the region for which the filtering process to makethe specified boundary less conspicuous is executed (i.e., the filteringregion). The filtering process is executed on the image data that hasbeen divided corresponding to the projector having the higherresolution. Stated another way, in this embodiment, the regionspecifying portion 303 specifies, as the filtering region, at least apart of the region of the image data that is output to the projector Bhaving the higher resolution than the projector A.

The region specifying portion 303 in this embodiment specifies a widthof the filtering region based on the difference between the resolutionsof the projectors. More specifically, when the difference in resolutionbetween the projectors on both the sides of the boundary is larger thana predetermined difference, the region specifying portion 303 sets thewidth of the filtering region to be larger than that set when thedifference in resolution is smaller than the predetermined difference.With such setting, when the difference in resolution is large, theresolution is more smoothly changed, and therefore the boundary can bemade less conspicuous. On the other hand, when the difference inresolution is small, the width of the filtering region is set to berelatively small, and therefore a processing load of the filteringprocess can be reduced.

For example, when the resolution of the projector B is three times theresolution of the projector A, the region specifying portion 303specifies 30% of the projected region of the projector B as thefiltering region. In other words, a part of the projected region of theprojector B, which corresponds to 30% thereof as measured from theboundary with respect to the projected region of the projector A, isspecified as the filtering region. Meanwhile, when the resolution of theprojector B is twice the resolution of the projector A, the regionspecifying portion 303 specifies 20% of the projected region of theprojector B as the filtering region. Alternatively, the width of thefiltering region may be held constant. Further, the region specifyingportion 303 may specify the width of the filtering region based on adistance through which an image is observed, or details of the image.

In step S305 (determination procedure), the coefficient determiningportion 304 determines the filter coefficient (smoothing parameter),which is used in the filtering process, based on both the direction ofthe boundary and the filtering region, which have been specified by theregion specifying portion 303. The coefficient determining portion 304in this embodiment determines the filter coefficient such that thepixels arrayed in the direction parallel to the boundary is morestrongly smoothed than the pixels arrayed in the direction perpendicularto the boundary.

FIGS. 4A and 4B are each an illustration to explain, by way of example,a method of determining the filter coefficient by the coefficientdetermining portion 304. FIG. 4A illustrates an example in which thetiled display is performed by arraying a projected region 401 of theprojector A having the lower resolution and a projected region 402 ofthe projector B having the higher resolution side by side. In such acase, the region specifying portion 303 specifies, as the filteringregion, at least a part (i.e., a filter applied region 403) of theprojected region 402 of the projector B having the higher resolution.Numeral 404 indicates a boundary line between the projected region 401of the projector A and the projected region 402 of the projector B.

FIG. 4B illustrates, in an enlarged scale, the filter applied region 403illustrated in FIG. 4A. As illustrated in FIG. 4B, the coefficientdetermining portion 304 determines a filter A 405, which takes a mean ofthree pixel values, as the filter coefficient for a region 407 in thefilter applied region 403, which is positioned nearer to the boundary404. Also, the coefficient determining portion 304 determines a filter B406, which takes a mean of two pixel values, as the filter coefficientfor a region 408 in the filter applied region 403, which is positionedfarther away from the boundary 404 than the region 407. Thus, thecoefficient determining portion 304 determines the filter coefficientsuch that pixels in the filtering region nearer to the boundary line aremore strongly smoothed than pixels in the filtering region farther awayfrom the boundary line.

Stated another way, the coefficient determining portion 304 determinesthe smoothing parameter such that those ones of pixels in the projectedregion of the projector B, which are each located at a second distancefrom the boundary between the projected regions of the projectors A andB, are more strongly smoothed than those ones of the relevant pixels,which are each located at a first distance that is longer than thesecond distance. Be it noted that the projector B has the higherresolution than the projector A.

Further, the coefficient determining portion 304 in this embodimentdetermines the filter coefficient such that the filtering process isexecuted on the pixels arrayed in the direction parallel to thedirection of the boundary, but the filtering process is not executed onthe pixels arrayed in the direction perpendicular to the direction ofthe boundary. Thus, the coefficient determining portion 304 determinesthe filter coefficient such that smoothing of the pixels arrayed in thedirection parallel to the direction of the boundary is more stronglyperformed than smoothing of the pixels arrayed in the directionperpendicular to the direction of the boundary. While this embodiment isdescribed in connection with an example in which the smoothing of thepixels arrayed in the direction perpendicular to the direction of theboundary is not performed, those pixels may be more weakly smoothed thanthe pixels arrayed in the direction parallel to the direction of theboundary.

Change of the resolution resulting from the filtering process in thisembodiment will be described with reference to FIGS. 5A to 5E. Each ofFIGS. 5A to 5E represents the relationship between the resolution ofeach projector and the coordinate in the direction (x direction)perpendicular to the boundary line. The resolution represented in FIGS.5A to 5E is a value corresponding to a maximum spatial frequency that isavailable for the projected image. As that value increases, an imagehaving a higher resolution can be projected.

FIG. 5A represents the resolution before the filtering process. Anabrupt change of the resolution between the projectors A and B ispresent at a boundary line 601, and the boundary line 601 is in aconspicuous state. FIG. 5B represents the resolution after the filteringprocess in the direction perpendicular to the boundary line. Because theresolution in the direction perpendicular to the boundary line does notso contribute to the recognition of the boundary line, the resolution inthe perpendicular direction on the same side as the projector B is notreduced or is slightly reduced. As illustrated in FIG. 5B, therefore,the change of the resolution after the filtering process in thedirection perpendicular to the boundary line remains large.

FIG. 5C represents the resolution after the filtering process in thedirection parallel to the boundary line. As described above, thedifference in resolution in the direction parallel to the boundary lineaffects the recognition of the boundary line to a larger extent than thedifference in resolution in the direction perpendicular to the boundaryline. Accordingly, the coefficient determining portion 304 determinesthe filter coefficient such that, in the projected region of theprojector B nearer to the boundary, the resolution in the directionparallel to the direction of the boundary becomes close to theresolution of the projector A. Further, the coefficient determiningportion 304 gradually increases the resolution of the projector B in thedirection parallel to the boundary line as a distance from the boundaryline increases, and then sets it to the same value as the originalresolution of the projector B in a region away from the boundary line inexcess of a filtering region width 602.

FIGS. 5D and 5E represent respectively the resolution in the x directionand the resolution in the y direction when the filtering process isexecuted regardless of direction. On the other hand, because the imageprocessing portion 201 in this embodiment holds the resolution in thedirection perpendicular to the direction of the boundary, i.e., in the xdirection, as it is, the resolution in the filtering region remains at ahigher level in comparison with the case where the filtering process isexecuted regardless of direction.

The methods of determining the filtering region and the filtercoefficient are not limited to the above-described ones. Thus, it isjust required that the pixel values in the direction parallel to theboundary line are more strongly smoothed than the pixel values in thedirection perpendicular to the boundary line.

In step S306 (smoothing procedure), the filtering portion 305 executesthe filtering process (smoothing process) by using both the filteringregion, which has been specified in step S304, and the filtercoefficient, which has been determined in step S305. More specifically,in step S306, the filtering portion 305 executes the smoothing processsuch that the pixels arrayed in the direction parallel to the boundaryis more strongly smoothed than the pixels arrayed in the directionperpendicular to the boundary. As the filtering process, the filteringportion 305 executes convolution integral of the original image data byusing the filter coefficient.

In step S307 (output procedure), the filtering portion 305 outputs theimage data after the filtering process to the projectors. Because theimage processing apparatus 101 according to this embodiment is describedas an apparatus separate from the projectors A and B, the filteringportion 305 outputs the image data to the projectors A and B. However,when the image processing apparatus 101 is incorporated in the projectorA, for example, the filtering portion 305 outputs the image data to theprojector B and to, e.g., the panel signal generating portion 102 in theprojector A. In other words, the filtering portion 305 outputs the imagedata after the filtering process to at least one of the pluralprojectors that are used to perform the multi-projection.

Part of the steps in FIG. 3 may be executed by using the function of,e.g., an OS (Operating System) or a known application.

Next, a result of the image processing executed for the boundary line inaccordance with the filtering process is described with reference toFIGS. 6A and 6B. FIG. 6A represents the resolution before the filteringprocess near the boundary line between the projectors A and B. Onesquare corresponds to one pixel. FIG. 6A indicates that the resolutionis abruptly changed on both the sides of a boundary line 701. FIG. 6Bindicates the resolutions of the projectors A and B after the filteringprocess near the boundary line. A region 702 represents a region towhich the filter A determined in step S305 is applied, and a region 703represents a region to which the filter B is applied. The filteringprocess (smoothing process) in this embodiment is executed by using amean value filter. Be it noted that the smoothing process may beexecuted by using a higher-frequency component cut filter or a centralvalue filter.

As illustrated in FIG. 6B, the resolution of the projector B after thefiltering process near the boundary line is reduced as a result of thefiltering process. In particular, the resolution in the directionparallel to the direction in which the boundary line extends is reducedin comparison with the resolution in the direction perpendicular to thedirection of the boundary line. Thus, according to the image processingapparatus 101 of this embodiment, an image with a high resolution in thedirection perpendicular to the direction of the boundary line can beprojected while the boundary between the projected regions of theprojectors having the different resolutions can be made lessconspicuous.

In this embodiment, the filtering process is described in connectionwith the case where the projectors A and B project the projected regionsarrayed side by side in the left-and-right direction to perform thetiled display, as illustrated in FIG. 1. However, embodiments are notlimited to the above-described one, and the filtering process can alsobe applied to the case where the projected regions of the projectors aredisplayed in an overlapped relation.

According to the image processing apparatus of this embodiment, asdescribed above, in the tiled display using a plurality of projectorshaving different resolutions, the boundary between the projected regionscan be made less conspicuous while reducing degradation of imagequality.

While the image processing apparatus of this embodiment is described inconnection with an example in which the multi-projection is performed byusing a plurality of projectors having different resolutions,embodiments are not limited to the above-described one. The presentinvention can be further applied to the case where the multi-projectionis performed by using a plurality of projectors having the sameresolution, but having different field angles, or the case where themulti-projection is performed by using a projector in which the keystonecorrection is executed and a projector in which the keystone correctionis not executed.

When the field angle is employed, the parameter acquiring portion 301acquires information regarding the resolution and the field angle fromeach projector in step S302 of FIG. 3. Further, in step S304, the regionspecifying portion 303 specifies the boundary between the projectedregions where the resolutions differ from each other, based on theresolutions, the region information, and the field angles, and thenspecifies the filtering region based on the specified result. Forexample, when the projectors have the same resolution, the resolution ofthe projector having a smaller field angle becomes relatively high.Accordingly, in step S305, the coefficient determining portion 304determines the filter coefficient (smoothing parameter) such that pixelsarrayed in the direction parallel to the boundary between the projectedregion of the projector having the smaller field angle and the projectedregion of the projector having the larger field angle are more stronglysmoothed than pixels arrayed in the direction perpendicular to theboundary.

When a correction amount for the keystone correction is employed, theparameter acquiring portion 301 acquires information regarding theresolution and the correction amount for the keystone correction fromeach projector in step S302 of FIG. 3. Further, in step S304, the regionspecifying portion 303 specifies the boundary between the projectedregions where the resolutions differ from each other, based on theresolutions, the correction amounts for the keystone correction, and theregion information, and then specifies the filtering region based on thespecified result. As an alternative, the image processing apparatus 101may specify the filtering region and determine the filter coefficient byemploying both the field angle and the correction amount for thekeystone correction.

Embodiment 2

Embodiment 2 is described primarily about different points in comparisonwith Embodiment 1. FIG. 7 is a block diagram to explain theconfiguration of multi-projection in Embodiment 2. This embodiment isdescribed in connection with the case where a projected region 1002 of aprojector B is surrounded by a projected region 1001 of a projector A.Be it noted that the projectors A and B in this embodiment are the sameas the projectors A and B in Embodiment 1.

FIG. 8 is a block diagram illustrating the configuration of an imageprocessing portion 1201 in Embodiment 2. While the image processingportion 201 in Embodiment 1 acquires the region information based on theuser inputs or the picked-up image, the image processing portion 1201 inthis embodiment acquires the region information based on the image datathat is output from an image dividing portion 1302 to a filteringportion 1305.

More specifically, the image dividing portion 1302 divides the imagedata based on information regarding the projected region, which isacquired from each projector, and further acquires the regioninformation by using the divided image data. For example, in FIG. 7, theimage dividing portion 1302 acquires the projected region informationnotifying that the projector A projects a region 1001 and the projectorB projects a region 1002. In an example of FIG. 7, the projected regioninformation acquired by the image dividing portion 1302 includescoordinate values of four apexes for specifying the region 1001,coordinate values of a center of the region 1002, and a radius of theregion 1002. The projected region information further includescorrespondence information for making the region 1001 and the projectorA correspondent to each other, and correspondence information for makingthe region 1002 and the projector B correspondent to each other. Theimage dividing portion 1302 having acquired the above-mentionedprojected region information divides the input image data into imagedata for the projector A and image data for the projector B.

An edge direction specifying portion 1306 specifies both the boundarybetween the projected regions where the resolutions differ from eachother and the direction of the boundary based on the divided image datathat has been output from the image dividing portion 1302. Processingexecuted by the edge direction specifying portion 1306 will be describedbelow.

FIGS. 9A to 9C illustrate an example of an image divided by the imagedividing portion 1302. FIG. 9A illustrates a projected region in whichthe projected regions of two projectors are combined with each other.The region 1001 represents the projected region of the projector A, andthe region 1002 represents the projected region of the projector B. Awhite region in FIG. 9B represents a region of the image data that isoutput from the image dividing portion 1302 for the projector A. Also, awhite region in FIG. 9C represents a region of the image data that isoutput from the image dividing portion 1302 for the projector B. Theimage dividing portion 1302 outputs the image data such that pixel valuedata for each of the white regions, illustrated in FIGS. 9B and 9C, takevalues corresponding to the input image data and pixel value data foreach of the black regions become 0. The edge direction specifyingportion 1306 specifies the boundary between the projected regions wherethe resolutions differ from each other and the direction of the boundarybased on both the image data output from the image dividing portion 1302and the resolutions acquired by the parameter acquiring portion 1301. Asan alternative, the edge direction specifying portion 1306 may acquirethe resolutions from the image data that is output from the imagedividing portion 1302.

The edge direction specifying portion 1306 in this embodiment specifiesthe boundary between the projected regions where the resolutions differfrom each other and the direction of the boundary in accordance with thefollowing method.

The edge direction specifying portion 1306 calculates differentialamounts in the x and y directions for an arbitrary pixel P(x, y) in theregion 1001:fx=P(x+1,y)−P(x,y)  (1)fy=P(x,y+1)−P(x,y)  (2)

Next, the edge direction specifying portion 1306 calculates intensity Iat an edge of the image according to the following formula.I(x,y)=√(fx×fx+fy×fy)  (3)

Next, the edge direction specifying portion 1306 compares the edgeintensity calculated according to the formula (3) with a threshold I0.

In the case of I(x, y)>I0, the edge direction specifying portion 1306determines that the pixel P(x, y) represents an edge portion. On theother hand, in the case of I(x, y)<=I0, the edge direction specifyingportion 1306 determines that the pixel P(x, y) does not represent anedge portion.

Subsequently, the edge direction specifying portion 1306 calculates anedge direction. The edge direction at the pixel P(x, y) is defined by;θ=arctan(fy/fx)  (4)The relationship between a boundary at the edge and θ is described withreference to FIGS. 10A and 10B. FIG. 10A illustrates the projectedregion in which the projected regions of the projectors A and B arecombined with each other, as in FIG. 9A. FIG. 10B illustrates, in anenlarged scale, a region 1006 in FIG. 10A. A line tangential to theboundary between the regions 1001 and 1002 at a certain pixel 1005 isprovided by a tangential line 1003. Angle information θ is calculated asan angle formed with respect to the direction perpendicular to thetangential line, i.e., to the boundary line. The edge directionspecifying portion 1306 in this embodiment executes the above-describedcalculations for all pixels in the region 1001.

The edge direction specifying portion 1306 outputs the angle informationθ to the coefficient determining portion 1304. The coefficientdetermining portion 1304 determines the filter coefficient (smoothingparameter) by using the angle information θ. More specifically, thecoefficient determining portion 1304 determines the smoothing parametersuch that pixels arrayed in the direction parallel to the direction ofthe boundary are more strongly smoothed than pixels arrayed in thedirection perpendicular to the direction of the boundary. Thecoefficient determining portion 1304 in this embodiment previouslystores the optimum filter coefficient for each value of the angleinformation θ. Then, the coefficient determining portion 1304 determinesthe filter coefficient in accordance with both the acquired angleinformation θ and the filter coefficient stored in advance. Be it notedthat a method of determining the filter coefficient is not limited tothe above-described one.

According to the image processing apparatus of this embodiment, asdescribed above, when an image based on image data is displayed by usinga plurality of projectors, the boundary between the projected regionscan be made less conspicuous while reducing degradation of imagequality.

While this embodiment is described in connection with the imageprocessing apparatus in which projectors each projecting an image onto ascreen from the front surface side displays the image, the presentinvention can also be applied to, e.g., a rear projection television,etc. Further, while this embodiment is described in connection with thecase where the image processing apparatus is separate from a projector,the image processing apparatus may be incorporated in the projector.

Embodiment 3

The first embodiment and the second embodiment have been described inconnection with the case of using two projectors, the present inventioncan also be applied to the case of using three or more projectors. Thisembodiment is described in connection with the case where themulti-projection is performed by using four projectors (i.e., projectorsA, B, C and D). For example, the projector D (fourth projector) projectsan image onto the projected region of the projector A (first projector)in FIG. 1, and the projector C (third projector) projects an image ontothe projected region of the projector B (second projector) in FIG. 1.The resolution of the projector B is highest and the resolutions of theother projectors decrease in the order of the projectors A, C and D.

In such an example, the filtering process is executed on a part of theprojected region of the projector B, which is positioned near theboundary with respect to the projected region of the projector A, and ona part of the projected region of the projector B, which is positionednear the boundary with respect to the projected region of the projectorC. Accordingly, two pixel values are calculated for one pixel in thepart (upper left corner) of the projected region of the projector B.More specifically, those two pixel values calculated are a first pixelvalue obtained with the filtering process, which is executed dependingon the direction of the boundary between the projected regions of theprojectors A and B, and a second pixel value obtained with the filteringprocess, which is executed depending on the direction of the boundarybetween the projected regions of the projectors B and C. When two ormore pixel values are calculated for one pixel, the image processingportion 201 in this embodiment determines, based on an image pattern ina boundary area, which one of the pixel values is to be employed.

To describe in more detail, the filtering portion 305 analyzes the imagein the projected region of the projector B and determines in which oneof two cases, i.e., the case where the smoothing process is morestrongly executed in the parallel direction and the case where thesmoothing process is more strongly executed in the perpendiculardirection, quality of the projected image is less degraded. Then, thefiltering portion 305 employs, based on the determination result, thepixel value calculated through the filtering process that has been morestrongly executed in the direction in which quality of the projectedimage is less degraded.

Stated another way, the image processing portion 201 in this embodimentperforms the multi-projection by using the projector A (firstprojector), the projector B (second projector), the projector C (thirdprojector), and the projector D (fourth projector). Further, the imageprocessing portion 201 analyzes the image data input to the imagedividing portion 302. When at least respective parts of a firstsmoothing region corresponding to the boundary between the projectedregions of the projectors A and B and a second smoothing regioncorresponding to the boundary between the projected regions of theprojectors B and C overlap with each other, the image processing portion201 determines the image data, which is to be output, as follows.Depending on the analyzed result of the image data, the image processingportion 201 outputs one of the image data that has been smootheddepending on the direction of the boundary between the projected regionsof the projectors A and B and the image data that has been smootheddepending on the direction of the boundary between the projected regionsof the projectors B and C. Degradation of the projected imageattributable to the filtering process can be reduced by determining thepixel value of the image data, which is to be output, based on the imagepattern, as described above.

However, the manner of selecting the pixel value to be employed is notlimited to the above-described one. For example, it is also possible toemploy a pixel value obtained through the filtering process that isexecuted depending on the direction of the boundary between theprojected region of one of the projector A and the projector C, whichhas a lower resolution, and the projected region of the projector B.

More specifically, when the resolution of the projector B is higher thanthat of the projector A and the resolution of the projector A is higherthan that of the projector C, the image processing portion 201 executesthe filtering process depending on the direction of the boundary betweenthe projected region of the projector C and the projected region of theprojector B. Then, the image processing portion 201 outputs the imagedata in which the pixel value obtained through the filtering processexecuted depending on the direction of the boundary between theprojectors B and C is provided as a pixel value in a region where thesmoothing regions overlap with each other.

Stated another way, when at least respective parts of the firstsmoothing region corresponding to the boundary between the projectedregions of the projectors A and B and the second smoothing regioncorresponding to the boundary between the projected regions of theprojectors B and C overlap with each other, the image processing portion201 determines the image data, which is to be output, as follows.Depending on the resolutions of the projectors A and C, the imageprocessing portion 201 outputs one of the image data that has beensmoothed depending on the direction of the boundary between theprojected regions of the projectors A and B and the image data that hasbeen smoothed depending on the direction of the boundary between theprojected regions of the projectors B and C.

The boundary corresponding to a larger difference in resolution can bemade less conspicuous by determining the image data, which is to beoutput, based on the respective resolutions of the projectors, asdescribed above. Further, by determining the output image data in such amanner, the processing can be simplified in comparison with the case ofdetermining the pixel value based on the image pattern. In addition, theimage processing portion 201 may determine the pixel value, which is tobe employed, after calculating two pixel values based on two smoothingparameters, or may calculate a pixel value after determining one of thetwo smoothing parameters.

Other Embodiments

The present invention can also be implemented by executing the followingprocess. More specifically, software (program) for implementing thefunctions of the above-described embodiments is supplied to a system oran apparatus via a network or some of various storage media, and acomputer (or a CPU, an MPU, etc.) in the system or the apparatus readsand executes the program. In an example, a storage media may have storedthereon, a program for a computer that executes a specifying procedureand an output procedure, where the specifying procedure is forspecifying a boundary between projected regions of a plurality ofprojectors having different resolutions based on the resolutions of theplurality of projectors and region information regarding the projectedregions of the plurality of projectors, and where the output procedureis for outputting, to the plurality of projectors, image data that issmoothed in accordance with the boundary specified in the smoothingprocedure. Here, the program may cause the computer to execute asmoothing procedure for executing smoothing such that pixels arrayed ina direction parallel to the boundary, which has been specified in thespecifying procedure, are more strongly smoothed than pixels arrayed ina direction perpendicular to the boundary.

According to the present invention, when one image is displayed byarraying the projected regions of the plurality of projectors, theboundary between the projected regions can be made less conspicuouswhile reducing degradation of image quality in the projected regions.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of International Application No.PCT/JP2009/069998, filed Nov. 27, 2009, which is hereby incorporated byreference herein in its entirety.

The invention claimed is:
 1. An image processing apparatus comprising:an acquiring unit configured to acquire resolutions of a first and asecond projectors and region information regarding projected regions ofthe first and second projectors; a specifying unit configured to specifya boundary between the projected regions of the first projector and thesecond projector having higher resolution than the first projector basedon the region information acquired by the acquiring unit; a smoothingunit configured to execute smoothing on a predetermined region within aprojected region of the second projector having higher resolution thanthe first projector based on the resolutions of the first and secondprojectors acquired by the acquiring unit such that a pixel obtainedbased on a first pixel by executing the smoothing has been affectedaccording to a second pixel than a third pixel, wherein the first pixelis included in the predetermined region, the first and second pixels arearrayed in a direction parallel to the boundary specified by thespecifying unit, and the first and third pixels are arrayed in adirection perpendicular to the boundary; and an output unit configuredto output image data that has been smoothed by the smoothing unit. 2.The image processing apparatus according to claim 1, wherein the outputunit outputs the image data, which has been smoothed by the smoothingunit, to at least one of a plurality of projectors.
 3. The imageprocessing apparatus according to claim 1, wherein the smoothing unitexecutes the smoothing such that pixels in a second projected region ofa second projector, which are located at a second distance from theboundary between a first projected region of a first projector having afirst resolution and the second projected region of the second projectorhaving a second resolution higher than the first resolution are morestrongly smoothed than pixels located at a first distance from theboundary, wherein the first distance is longer than the second distance.4. The image processing apparatus according to claim 1, wherein theacquiring unit acquires resolutions of a plurality of projectors basedon notices from the plurality of projectors.
 5. The image processingapparatus according to claim 1, further comprising a designationinformation input unit configured to input designation information todesignate projected regions of a plurality of projectors, wherein theacquiring unit acquires the region information regarding the projectedregions of the plurality of projectors based on the designationinformation input through the designation information input unit.
 6. Theimage processing apparatus according to claim 1, further comprising animage input unit configured to input an image picked up by an imagepickup unit, wherein the acquiring unit acquires the region informationregarding the projected regions of a plurality of projectors based onthe picked-up image input through the image input unit.
 7. The imageprocessing apparatus according to claim 1, further comprising ananalyzing unit configured to analyze image data input to the imageprocessing apparatus, wherein, when at least respective parts of a firstsmoothing region corresponding to a first boundary between a projectedregion of a first projector and a projected region of a second projectorhaving a higher resolution than the first projector and a secondsmoothing region corresponding to a second boundary between theprojected region of the second projector and a projected region of athird projector having a lower resolution than the second projectoroverlap with each other to form an overlapped region, the output unitoutputs, depending on a result analyzed by the analyzing unit, one ofimage data obtained by smoothing the overlapped region in accordancewith a direction of the first boundary and image data obtained bysmoothing the overlapped region in accordance with a direction of thesecond boundary.
 8. The image processing apparatus according to claim 1,wherein when at least respective parts of a first smoothing regioncorresponding to a first boundary between a projected region of a firstprojector and a projected region of a second projector having a higherresolution than the first projector and a second smoothing regioncorresponding to a second boundary between the projected region of thesecond projector and a projected region of a third projector having alower resolution than the second projector overlap with each other toform an overlapped region, the output unit outputs, depending onrespective resolutions of the first and third projectors, one of imagedata obtained by smoothing the overlapped region in accordance with adirection of the first boundary and image data obtained by smoothing theoverlapped region in accordance with a direction of the second boundary.9. An image processing method for an image processing apparatus, theimage processing method comprising: acquiring resolutions of a first anda second projectors and region information regarding projected regionsof the first and second projectors; specifying a boundary between theprojected regions of the first projector and the second projector havinghigher resolution than the first projector based on the acquired regioninformation; executing smoothing on a predetermined region within aprojected region of the second projector having higher resolution thanthe first projector based on the acquired resolutions of the first andsecond projectors such that a pixel obtained based on a first pixel byexecuting the smoothing has been affected according to a second pixelthan a third pixel, wherein the first pixel is included in thepredetermined region, the first and second pixels are arrayed in adirection parallel to the specified boundary, and the first and thirdpixels are arrayed in a direction perpendicular to the boundary; andoutputting image data that has been smoothed.
 10. A non-transitorystorage medium storing a program to cause an image processing apparatusto perform a method, the method comprising: acquiring resolutions of afirst and a second projectors and region information regarding projectedregions of the first and second projectors; specifying a boundarybetween the projected regions of the first projector and the secondprojector having higher resolution than the first projector based on theacquired region information; executing smoothing on a predeterminedregion within a projected region of the second projector having higherresolution than the first projector based on the acquired resolutions ofthe first and second projectors such that a pixel obtained based on afirst pixel by executing the smoothing has been affected according to asecond pixel than a third pixel, wherein the first pixel is included inthe predetermined region, the first and second pixels are arrayed in adirection parallel to the specified boundary, and the first and thirdpixels are arrayed in a direction perpendicular to the boundary; andoutputting image data that has been smoothed.
 11. The image processingapparatus according to claim 1, wherein the smoothing unit executessmoothing for the predetermined region such that a resolution of thepixels in the direction parallel to the boundary is lower than aresolution of the pixels in the direction perpendicular to the boundary.12. The image processing apparatus according to claim 1, wherein thesmoothing unit executes mean value processing for a predetermined numberof pixels arrayed in the direction parallel to the boundary and does notexecute the mean value processing for the predetermined number of pixelsarrayed in the direction perpendicular to the boundary.
 13. The methodaccording to claim 9, wherein smoothing includes executing the smoothingsuch that pixels in a second projected region of a second projector,which are located at a second distance from the boundary between a firstprojected region of a first projector having a first resolution and thesecond projected region of the second projector having a secondresolution higher than the first resolution are more strongly smoothedthan pixels located at a first distance from the boundary, wherein thefirst distance is longer than the second distance.
 14. The methodaccording to claim 9, further comprising analyzing image data input tothe image processing apparatus, wherein, when at least respective partsof a first smoothing region corresponding to a first boundary between aprojected region of a first projector and a projected region of a secondprojector having a higher resolution than the first projector and asecond smoothing region corresponding to a second boundary between theprojected region of the second projector and a projected region of athird projector having a lower resolution than the second projectoroverlap with each other to form an overlapped region, outputtingincludes outputting, depending on an analyzed result, one of image dataobtained by smoothing the overlapped region in accordance with adirection of the first boundary and image data obtained by smoothing theoverlapped region in accordance with a direction of the second boundary.15. The method according to claim 9, wherein when at least respectiveparts of a first smoothing region corresponding to a first boundarybetween a projected region of a first projector and a projected regionof a second projector having a higher resolution than the firstprojector and a second smoothing region corresponding to a secondboundary between the projected region of the second projector and aprojected region of a third projector having a lower resolution than thesecond projector overlap with each other to form an overlapped region,outputting includes outputting, depending on respective resolutions ofthe first and third projectors, one of image data obtained by smoothingthe overlapped region in accordance with a direction of the firstboundary and image data obtained by smoothing the overlapped region inaccordance with a direction of the second boundary.
 16. The methodaccording to claim 9, wherein smoothing includes executing smoothing forthe predetermined region such that a resolution of the pixels in thedirection parallel to the boundary is lower than a resolution of thepixels in the direction perpendicular to the boundary.
 17. The methodaccording to claim 9, wherein smoothing includes executing mean valueprocessing for a predetermined number of pixels arrayed in the directionparallel to the boundary and does not execute the mean value processingfor the predetermined number of pixels arrayed in the directionperpendicular to the boundary.
 18. The non-transitory storage mediumaccording to claim 10, wherein smoothing includes executing thesmoothing such that pixels in a second projected region of a secondprojector, which are located at a second distance from the boundarybetween a first projected region of a first projector having a firstresolution and the second projected region of the second projectorhaving a second resolution higher than the first resolution are morestrongly smoothed than pixels located at a first distance from theboundary, wherein the first distance is longer than the second distance.19. The non-transitory storage medium according to claim 10, furthercomprising analyzing image data input to the image processing apparatus,wherein, when at least respective parts of a first smoothing regioncorresponding to a first boundary between a projected region of a firstprojector and a projected region of a second projector having a higherresolution than the first projector and a second smoothing regioncorresponding to a second boundary between the projected region of thesecond projector and a projected region of a third projector having alower resolution than the second projector overlap with each other toform an overlapped region, outputting includes outputting, depending onan analyzed result, one of image data obtained by smoothing theoverlapped region in accordance with a direction of the first boundaryand image data obtained by smoothing the overlapped region in accordancewith a direction of the second boundary.
 20. The non-transitory storagemedium according to claim 10, wherein when at least respective parts ofa first smoothing region corresponding to a first boundary between aprojected region of a first projector and a projected region of a secondprojector having a higher resolution than the first projector and asecond smoothing region corresponding to a second boundary between theprojected region of the second projector and a projected region of athird projector having a lower resolution than the second projectoroverlap with each other to form an overlapped region, outputtingincludes outputting, depending on respective resolutions of the firstand third projectors, one of image data obtained by smoothing theoverlapped region in accordance with a direction of the first boundaryand image data obtained by smoothing the overlapped region in accordancewith a direction of the second boundary.
 21. The non-transitory storagemedium according to claim 10, wherein smoothing includes executingsmoothing for the predetermined region such that a resolution of thepixels in the direction parallel to the boundary is lower than aresolution of the pixels in the direction perpendicular to the boundary.22. The non-transitory storage medium according to claim 10, whereinsmoothing includes executing mean value processing for a predeterminednumber of pixels arrayed in the direction parallel to the boundary anddoes not execute the mean value processing for the predetermined numberof pixels arrayed in the direction perpendicular to the boundary.